{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "KjTHAV8FgEza"
      },
      "outputs": [],
      "source": [
        "# Copyright 2025 Google LLC\n",
        "#\n",
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "#     https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UdL4uvQQs76x"
      },
      "source": [
        "# Veo 2 Advanced Controls\n",
        "\n",
        "<table align=\"left\">\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/generative-ai/blob/main/vision/getting-started/veo2_advanced_controls.ipynb\">\n",
        "      <img src=\"https://www.gstatic.com/pantheon/images/bigquery/welcome_page/colab-logo.svg\" alt=\"Google Colaboratory logo\"><br> Run in Colab\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fgenerative-ai%2Fmain%2Fvision%2Fgetting-started%2Fveo2_advanced_controls.ipynb\">\n",
        "      <img width=\"32px\" src=\"https://lh3.googleusercontent.com/JmcxdQi-qOpctIvWKgPtrzZdJJK-J3sWE1RsfjZNwshCFgE_9fULcNpuXYTilIR2hjwN\" alt=\"Google Cloud Colab Enterprise logo\"><br> Run in Colab Enterprise\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/generative-ai/main/vision/getting-started/veo2_advanced_controls.ipynb\">\n",
        "      <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\"><br> Open in Vertex AI Workbench\n",
        "    </a>\n",
        "  </td>    \n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://github.com/GoogleCloudPlatform/generative-ai/blob/main/vision/getting-started/veo2_advanced_controls.ipynb\">\n",
        "      <img width=\"32px\" src=\"https://raw.githubusercontent.com/primer/octicons/refs/heads/main/icons/mark-github-24.svg\" alt=\"GitHub logo\"><br> View on GitHub\n",
        "    </a>\n",
        "  </td>\n",
        "</table>\n",
        "\n",
        "<div style=\"clear: both;\"></div>\n",
        "\n",
        "<b>Share to:</b>\n",
        "\n",
        "<a href=\"https://www.linkedin.com/sharing/share-offsite/?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/vision/getting-started/veo2_advanced_controls.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/8/81/LinkedIn_icon.svg\" alt=\"LinkedIn logo\">\n",
        "</a>\n",
        "\n",
        "<a href=\"https://bsky.app/intent/compose?text=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/vision/getting-started/veo2_advanced_controls.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/7/7a/Bluesky_Logo.svg\" alt=\"Bluesky logo\">\n",
        "</a>\n",
        "\n",
        "<a href=\"https://twitter.com/intent/tweet?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/vision/getting-started/veo2_advanced_controls.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/5a/X_icon_2.svg\" alt=\"X logo\">\n",
        "</a>\n",
        "\n",
        "<a href=\"https://reddit.com/submit?url=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/vision/getting-started/veo2_advanced_controls.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://redditinc.com/hubfs/Reddit%20Inc/Brand/Reddit_Logo.png\" alt=\"Reddit logo\">\n",
        "</a>\n",
        "\n",
        "<a href=\"https://www.facebook.com/sharer/sharer.php?u=https%3A//github.com/GoogleCloudPlatform/generative-ai/blob/main/vision/getting-started/veo2_advanced_controls.ipynb\" target=\"_blank\">\n",
        "  <img width=\"20px\" src=\"https://upload.wikimedia.org/wikipedia/commons/5/51/Facebook_f_logo_%282019%29.svg\" alt=\"Facebook logo\">\n",
        "</a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lUCUMoTmN_lJ"
      },
      "source": [
        "| | |\n",
        "|-|-|\n",
        "|Author(s) | [Katie Nguyen](https://github.com/katiemn) |"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rDjAqcgigwdX"
      },
      "source": [
        "## Overview\n",
        "\n",
        "### Veo 2\n",
        "\n",
        "Veo 2 on Vertex AI brings Google's video generation capabilities to application developers. It's capable of creating videos with astonishing detail that simulate real-world physics across a wide range of visual styles.\n",
        "\n",
        "In this tutorial, you will learn how to use the Google Gen AI SDK for Python to interact with Veo 2 advanced control features, including:\n",
        "\n",
        "- Frame interpolation\n",
        "- Video extend\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "v2_iOv5uhXVg"
      },
      "source": [
        "## Get started"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "K4uerc9Xhf1f"
      },
      "source": [
        "### Install Google Gen AI SDK for Python"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rJyFNKoQhiwF"
      },
      "outputs": [],
      "source": [
        "%pip install --upgrade --quiet google-genai"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GWYnCW0-h6HI"
      },
      "source": [
        "### Authenticate your notebook environment (Colab only)\n",
        "\n",
        "If you are running this notebook on Google Colab, run the following cell to authenticate your environment."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "bqz5LUG6h8fA"
      },
      "outputs": [],
      "source": [
        "import sys\n",
        "\n",
        "if \"google.colab\" in sys.modules:\n",
        "    from google.colab import auth\n",
        "\n",
        "    auth.authenticate_user()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LVrasKoriKZn"
      },
      "source": [
        "### Import libraries"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yxBkUEqdiB1g"
      },
      "source": [
        "### Set Google Cloud project information and create client\n",
        "\n",
        "To get started using Vertex AI, you must have an existing Google Cloud project and [enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com).\n",
        "\n",
        "Learn more about [setting up a project and a development environment](https://cloud.google.com/vertex-ai/docs/start/cloud-environment)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "GtjPBmYHiEfx"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "import time\n",
        "\n",
        "import matplotlib.image as img\n",
        "import matplotlib.pyplot as plt\n",
        "from IPython.display import Video, display\n",
        "from google import genai\n",
        "from google.genai import types\n",
        "\n",
        "# fmt: off\n",
        "PROJECT_ID = \"[your-project-id]\"  # @param {type: \"string\", placeholder: \"[your-project-id]\", isTemplate: true}\n",
        "# fmt: on\n",
        "if not PROJECT_ID or PROJECT_ID == \"[your-project-id]\":\n",
        "    PROJECT_ID = str(os.environ.get(\"GOOGLE_CLOUD_PROJECT\"))\n",
        "\n",
        "LOCATION = os.environ.get(\"GOOGLE_CLOUD_REGION\", \"us-central1\")\n",
        "\n",
        "client = genai.Client(vertexai=True, project=PROJECT_ID, location=LOCATION)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qD_bwA9hiMzL"
      },
      "source": [
        "### Define a helper function to display media"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "GUrEwbvFiPhJ"
      },
      "outputs": [],
      "source": [
        "def show_video(video):\n",
        "    if isinstance(video, str):\n",
        "        file_name = video.split(\"/\")[-1]\n",
        "        !gsutil cp {video} {file_name}\n",
        "        display(Video(file_name, embed=True, width=600, height=400))\n",
        "    else:\n",
        "        with open(\"sample.mp4\", \"wb\") as out_file:\n",
        "            out_file.write(video)\n",
        "        display(Video(\"sample.mp4\", embed=True, width=600, height=400))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2jaSOOadiUj6"
      },
      "source": [
        "### Load the video generation model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "id": "APRfTklCiYR2"
      },
      "outputs": [],
      "source": [
        "video_model = \"veo-2.0-generate-001\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_1-VPOexGWe8"
      },
      "source": [
        "## Video generation with Veo 2 advanced controls"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "P604bVwQeDPN"
      },
      "source": [
        "### Frame interpolation\n",
        "\n",
        "With frame interpolation, you can specify the first and last frame of the video clip, and Veo will generate the rest.\n",
        "\n",
        "#### Load the images\n",
        "\n",
        "Download the first and last frames that you will use in the request. If you would like to use local images, simply comment out the `wget` command and edit the `first_frame` and `last_frame` variables.\n",
        "\n",
        "**Note:** This feature tends to work best when the starting and ending frames are similar in nature."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "5-fpFTzwefcH"
      },
      "outputs": [],
      "source": [
        "# Download first frame\n",
        "!wget https://storage.googleapis.com/cloud-samples-data/generative-ai/image/cookies.png\n",
        "# fmt: off\n",
        "first_frame = \"cookies.png\"  # @param {type: 'string'}\n",
        "\n",
        "# Download last frame\n",
        "!wget https://storage.googleapis.com/cloud-samples-data/generative-ai/image/cookies-milk.png\n",
        "last_frame = \"cookies-milk.png\"  # @param {type: 'string'}\n",
        "# fmt: on\n",
        "\n",
        "# Display the images\n",
        "fig, axis = plt.subplots(1, 2, figsize=(12, 6))\n",
        "axis[0].imshow(img.imread(first_frame))\n",
        "axis[0].set_title(\"First Frame\")\n",
        "axis[1].imshow(img.imread(last_frame))\n",
        "axis[1].set_title(\"Last Frame\")\n",
        "for ax in axis:\n",
        "    ax.axis(\"off\")\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yKR0riV_K5nT"
      },
      "source": [
        "With Veo 2, you have the option to generate 720p videos from a text prompt and/or input image(s). In order to generate a video in the following sample, specify the following info:\n",
        "- **Prompt:** A description of the video you would like to see. Since you're starting with images you can also leave the prompt field empty.\n",
        "- **Aspect ratio:** Select either 16:9 or 9:16.\n",
        "- **Number of videos:** Set this value to 1 or 2.\n",
        "- **Video duration:** Can be 5, 6, 7, or 8 seconds.\n",
        "- **Person generation:** Set to `allow_adult` or `dont_allow`.\n",
        "- **Prompt enhancement:** The `veo-2.0-generate-001` model offers the option to enhance your provided prompt. To utilize this feature, set `enhance_prompt` to True. A new, detailed prompt will be created from your original one to help generate higher quality videos that better adhere to your prompt's intent."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3YzkQI5JekEh"
      },
      "outputs": [],
      "source": [
        "# fmt: off\n",
        "prompt = \"a hand reaches in and places a glass of milk next to the plate of cookies\"  # @param {type: 'string'}\n",
        "# fmt: on\n",
        "aspect_ratio = \"9:16\"  # @param [\"16:9\", \"9:16\"]\n",
        "person_generation = \"allow_adult\"  # @param [\"allow_adult\", \"dont_allow\"]\n",
        "enhance_prompt = True  # @param {type: 'boolean'}\n",
        "\n",
        "operation = client.models.generate_videos(\n",
        "    model=video_model,\n",
        "    prompt=prompt,\n",
        "    image=types.Image.from_file(location=first_frame),\n",
        "    config=types.GenerateVideosConfig(\n",
        "        aspect_ratio=aspect_ratio,\n",
        "        last_frame=types.Image.from_file(location=last_frame),\n",
        "        number_of_videos=1,\n",
        "        duration_seconds=7,\n",
        "        person_generation=person_generation,\n",
        "        enhance_prompt=enhance_prompt,\n",
        "    ),\n",
        ")\n",
        "\n",
        "while not operation.done:\n",
        "    time.sleep(5)\n",
        "    operation = client.operations.get(operation)\n",
        "    print(operation)\n",
        "\n",
        "if operation.response:\n",
        "    show_video(operation.result.generated_videos[0].video.video_bytes)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XPu4QrPAGanp"
      },
      "source": [
        "### Video extend\n",
        "\n",
        "In this next example, you'll use Veo to extend an existing video. You'll use the same parameters as before. However, the following are configured differently in this instance:\n",
        "- **Input video:** Since you're starting with a video, specify a Cloud Storage video location in the `video_gcs` variable.\n",
        "- **Output video:** As these videos will be bigger, you'll save your output video in Cloud Storage. Include the Cloud Storage bucket in the `output_gcs` field.\n",
        "- **Video duration:** Can be 4, 5, 6, or 7 seconds of extended video.\n",
        "\n",
        "**Safety:** All Veo videos include [SynthID](https://deepmind.google/technologies/synthid/), which embeds a digital watermark directly into the AI-generated video."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ylUHNer1GleN"
      },
      "outputs": [],
      "source": [
        "prompt = \"a butterfly flies in and lands on the flower\"  # @param {type: 'string'}\n",
        "# fmt: off\n",
        "video_gcs = \"gs://cloud-samples-data/generative-ai/video/flower.mp4\"  # @param {type: 'string'}\n",
        "# fmt: on\n",
        "aspect_ratio = \"16:9\"  # @param [\"16:9\", \"9:16\"]\n",
        "output_gcs = \"gs://\"  # @param {type: 'string'}\n",
        "person_generation = \"allow_adult\"  # @param [\"allow_adult\", \"dont_allow\"]\n",
        "enhance_prompt = True  # @param {type: 'boolean'}\n",
        "\n",
        "operation = client.models.generate_videos(\n",
        "    model=video_model,\n",
        "    prompt=prompt,\n",
        "    video=types.Video(uri=video_gcs),\n",
        "    config=types.GenerateVideosConfig(\n",
        "        aspect_ratio=aspect_ratio,\n",
        "        output_gcs_uri=output_gcs,\n",
        "        number_of_videos=1,\n",
        "        duration_seconds=7,\n",
        "        person_generation=person_generation,\n",
        "        enhance_prompt=enhance_prompt,\n",
        "    ),\n",
        ")\n",
        "\n",
        "while not operation.done:\n",
        "    time.sleep(5)\n",
        "    operation = client.operations.get(operation)\n",
        "    print(operation)\n",
        "\n",
        "if operation.response:\n",
        "    show_video(operation.result.generated_videos[0].video.uri)"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "veo2_advanced_controls.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
